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Abstract. We prove a general rigidity theorem for represented semi-simple Lie groups. 
The theorem is used to show that the adjoint variety of a complex simple Lie algebra g 
(the unique minimal G orbit in Pg) is extrinsically rigid to third order (with the exception 
of = Ol). 

In contrast, we show that the adjoint variety of SL^C and the Segre product Seg{¥^ xf") 
are flexible at order two. In the SL3C example we discuss the relationship between the 
extrinsic projective geometry and the intrinsic path geometry. 

We extend machinery developed by Hwang and Yamaguchi, Se-ashi, Tanaka and others 
to reduce the proof of the general theorem to a Lie algebra cohomology calculation. The 
proofs of the flexibility statements use exterior differential systems techniques. 



1. Introduction 

1.1. History and statement of the problem. The problem of determining the projective 
(or extrinsic) rigidity of varieties X C CP^ = P''^ dates back to Monge and has been 
studied by Fubini [8], Griffiths and Harris [9] and others. The problem may be stated 
informally as fohows: given a homogeneous variety Z = G/P C P?7 = P^ and an unknown 
variety Y C Pl¥ = P*-^, how many derivatives do we need to take at a general point of 
Y to determine whether or not Y is projectively equivalent to Z? More precisely, there 
is a sequence of relative differential invariants of a projective variety X C P^, defined at 
a smooth point x ^ X (the Fubini forms, see §3.1) that encode the extrinsic geometric 
information of X. Let T^X and N^X = Tx^^ /T^X denote the tangent and normal spaces 
to X at X € X. Then the k-th Fubini form i^^ at x is an element of S^T*X ® N^X modulo 
an equivalence relation. If we (locally) express X as graph in P^ over its embedded tangent 
space, then the Fubini forms F]^ ^ ^-re defined by the degree k coefficients in the Taylor 
series expansion of X about x, modulo the equivalence relation given by other choices of 
first-order adapted local coordinates. 

Definition 1.1. Let X C P{7 he a projective variety and let x ^ X he a general point. 

olfYc ¥W is another variety such that for a general point y (z Y , there exist bijective 
linear maps 4>t ■ T*Y T*X, (p^ : NyY N^X , such that the induced linear maps 

{<PtT^ (g, (t)N ■■ S^T*Y NyY ^ S^T*X ® N^X 

take (equivalence classes of) Fuhini forms ofY to (equivalence classes of) Fuhini forms 
of X for all i < k, then we say that Y agrees with X to order k. 
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o We say X is rigid at order k if whenever a variety Y agrees with X to order k, there 

exists a linear map ^ -.W such that ^{Y) = X. 
o When X is not rigid at order k, hut the set of distinct Y agreeing with X to order k is 

finite dimensional, we say X is quasi-rigid at order k. 
o Otherwise we say X is flexible at order k. 

Remark. In this paper we discuss the rigidity problem in ¥U. It is also natural to study 
rigidity when the ambient space is a homogeneous variety G'/P' and the model variety 
is a G-variety. See, for example, the study of Schubert varieties in compact Hermitian 
symmetric spaces in [3]. 

None of the examples in this paper are quasi-rigid. The paper [3] contains quasi-rigid 
examples for a closely related problem. 

Projective space embedded as a linear subspace P" C is easily seen to be rigid 
at order two. Fubini [8] proved that the smooth n-dimensional quadric hypersurface is 
rigid at order three when n > 1. When n = 1 Monge proved that the quadric is rigid 
at order five (cf. [15, Ex. 2.26]). Griffiths and Harris [9] conjectured that the Segre 
variety Seg{¥'^ x P^) C P^ is rigid at order two, and the conjecture was proven in [16]. 
Then in [17] it was shown that any irreducible, rank two, compact Hermitian symmetric 
space (CHSS) in its minimal homogeneous embedding (except for the quadric hypersurface) 
is rigid at order two. Hwang and Yamaguchi [10] then solved the rigidity problem for 
all homogeneously embedded irreducible CHSS: other than the quadric hypersurface, an 
irreducible, homogeneously embedded CHSS with osculating sequence of length / is rigid 
at order /. (The length of the osculating filtration of a CHSS in its minimal homogeneous 
embedding is equal to its rank.) 

The next simplest homogeneous varieties are the adjoint varieties C Pg, the homo- 
geneous complex contact manifolds. These are the unique closed orbits in Pg of the adjoint 
action of the associated complex simple Lie group G. When G = C„, = V2(P^''-^) is 
the quadratic Veronese variety which is CHSS for the larger group ^2n-i- In [16] ^2(1?"*), 
m > 1, was shown to be rigid at order three. This variety is flexible at order two since its 
second fundamental form is generic. The quadratic Veronese varieties have vanishing third 
and fourth order Fubini forms. In [22] it was observed that the third and fourth order Fubini 
invariants are non-zero for the other adjoint varieties. This led the authors to speculate 
that the other adjoint varieties would be rigid at order four, but not three. Robles [23] then 
showed that Z^J" is rigid at order three. (Note that, with the exceptions of a,i and Cn, the 
adjoint representations of the complex simple Lie algebras are fundamental.) 

1.2. Results. 

Theorem 1.2. The adjoint variety of a complex simple Lie algebra Q, Q ^ di, is rigid at 

order three. 

Theorem 1.2 is a consequence of Theorem 1.7 (g 7^ a„). Lemma 7.3 (when g = o„, n > 2) 
and Proposition 5.2. The case of A2 was resolved in [23] and we give a different proof in 
§9.5. 

Theorem 1.3. Assume n, r > 1. 
(a) The Veronese varieties Vd{P^) are rigid at order d+l. 
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(b) The Veronese embedding s of the quadric hypersurfaces Vd{Q^) are rigid at order 2d+l. 

(c) For integers 1 = ai < 02 < • • • < Or, the Segre variety Seg{F"-'^ x P"^ x • • • x P"'') is 
rigid at order r + 1. 

Theorem 1.3 is proven in §10.1. (The proof references calculations in §7.3.) 

Remark. If all the integers in Theorem 1.3 (c) satisfy Oj > 1, then Seg is rigid at order r, 
c.f. [10]. The third-order rigidity of Seg{F'^ x P^) C P^ is a consequence of Fubini's theorem 

[8], as it is a quadric hyper surf ace. 

We apply standard exterior differential systems (EDS) techniques, guided by representa- 
tion theory, to establish the following two flexibility results in §8 and §9, respectively. We 
remark that the calculations for Theorem 1.5 are quite long. 

Theorem 1.4. The Segre variety Seg{F^ x P") C p2"+i^ n> 1, is flexible at order two. It 
is rigid at order three. 

Theorem 1.4 is proved in §8. In [17] it was mistakenly remarked that Seg{F^ x P") was 
rigid at order two. The reasons for this error are explained in §8. Notice that Theorem 
1.4 implies that Theorem 1.3 (c) is sharp, and the rigidity statement is a consequence of 
Theorem 1.3 (c). 

Theorem 1.5. The adjoint variety = Flagi 2{C^) is flexible at order two. The general 
integral manifold gives rise to a non-flat path geometry. There are also integral manifolds 
that are projectively inequivalent to the homogeneous model and give rise to flat path ge- 
ometries. 

Theorem 1.5 is proved in §9. We actually prove a stronger flexibility result. In §4.2 
we define the (/_i, J-i) system associated to any homogeneously embedded homogeneous 
variety. (This system is more restrictive than the second order Fubini system.) A parabolic 
P C G determines a Z-grading of q and all g-modules, including gi{U), where G/P C PC/. 
For p = 0,-1, we define the (Ip,Jp) exterior differential system by restricting the the 
component of the Maurer-Cartan form of GL{U) taking values in the q-ik homogeneous 
part of q\{U) to take values in g C 0t(^7), for q < p. See §4.2 for a detailed description. 

While the {Ip, Jp) systems are natural from the point of view of representation theory, 
except in the case of generalized cominuscule representations, they do not lead one to Lie 
algebra cohomology. To rectify this, we define a class of filtered EDS as follows: Define 
(/f,ri) to be the {p + l)-filtered EDS on GL{U) with the same independence condition 
as the (/p, Jp) system but only requiring integral manifolds i : M — >■ GL{U) to satisfy 
It-i-^m) = where the filtration T'^M C T'^M C ■ ■ ■ C T^'^M = ■■■ = TM on 

TM is induced from the grading on TGL{U). See §6 for more details. 

For the adjoint varieties, the (/-i, J-i) system implies second order agreement plus par- 
tial third order agreement. The integral manifolds are bundles C GL{U) over varieties 
Y C FU. The EDS imposes the condition that the base varieties Y all have contact hyper- 
plane distributions, and that they support an intrinsic parabolic geometry. When G = SL^C 
the parabolic geometry is the incidence space for a path geometry in the plane. 

The variety Z'^J^'^ = Flag i 2{C^) is the homogeneous model for the incidence space of a 
path geometry in the plane. This geometry has been studied extensively. The equivalence 
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problem was solved by Cartan [6]. Intrinsically, on a three-fold X equipped with two foli- 
ations by curves whose tangent lines span a contact distribution, there are two differential 
invariants, Ji and J2, which measure the failure of X to be locally equivalent to the ho- 
mogeneous model. We show that there are three distinct types of integral manifolds. The 
first class consists only of the standard homogeneous model. The second class depends (in 
the language of Cartan) on four functions of one variable. Despite this flexibility, all the 
examples in the second class are intrinsically flat. (One can think by analogy of the surfaces 
of zero Gauss curvature in Euclidean 3-space, which depend on functions of one variable.) 
The third class of integral manifolds depends roughly on two functions of two variables. 
Here the intrinsic invariants are nonzero. 

As mentioned above, the (/_i, J_i) system is more restrictive than specifying second 
order agreement. In the case of the adjoint varieties it is less restrictive than specifying 
third order agreement. Theorems 1.2 and 1.3 are corollaries of Theorem 1.7 below. Loosely 
speaking, adjoint varieties for simple Lie algebras other than 5(2 and 5(3 are rigid to an 
order between two and three. In [22] it was observed that adjoint varieties are not rigid 
to order two, but only one non-isomorphic variety that agreed to order two was known: 
a "parabola" which had the same second fundamental form of the corresponding adjoint 
variety and all other differential invariants zero. These parabolas are not equipped with a 
contact structure. 

Definition 1.6. A homogeneous variety Z = G/P C Pt/ is rigid for the {Ip,Jp) system 
(or rigid for the (//,fi) system that is defined in %6) if the only integral manifolds of the 
system on SL(U) correspond to conjugates of G C SL(U). 

In this case the underlying projective varieties are all projcctivcly equivalent to Z. Any 
variety rigid for the {Ip,Jp) system (resp. the (/p,f^) system) is automatically rigid for the 

(Ip+uJp+i) system (resp. the (if^^n) system). Moreover, any variety rigid for the (1^, fi) 
system is automatically rigid for the (Ip, Jp) system. 

Let G have rank r, let / C {1, ...,r} and write P = Pj C G for the parabolic subgroup 
obtained by deleting negative root spaces corresponding to roots having a nonzero coefficient 
on any of the simple roots ctj, i G /. 

Theorem 1.7. Let G be a complex semi-simple group and let Z = G/P C FU be a ho- 
mogeneously embedded homogeneous variety (i.e., the orbit of a highest weight line in FU ). 
Assume that no factor of Z corresponds to a quadric hypersurface or A^/Pi, with 1 or n 
in I. Then Z is rigid for the {iLi,^)- system. 

The hypotheses above exclude the adjoint variety Z^ , where I = {l,n}. Nonetheless, 

Theorem 1.8. The adjoint variety Z^^ C F{an), forn > 2, is rigid for the system. 

Theorem 1.9. The Veronese variety t;rf(P") C P(5'^C"+^) = FU^^, n>2, and the adjoint 
variety Z^ C Pa2 are rigid for the [Iq, Jq) system. 

Theorems 1.7, 1.8 and 1.9 are proven in §7.3. 

1.3. Reduction to Lie algebra cohomology. Given a homogeneous variety Z = G/P C 
FU = and a point x E Z, one obtains two filtrations of U. The first is the osculating 
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filtration (see §4.1) which any manifold in P has. This filtration corresponds to the spans 
of successive derivatives at x of curves on Z. The second is the grading filtration which 
is given in terms of Lie algebra data (see §4.2). The two filtrations coincide if and only if 
X is a homogeneously embedded compact Hermitian symmetric space (CHSS). There are 
natural exterior differential systems (EDS) on GL{U) associated to each of these filtrations. 

In [10], Hwang and Yamaguchi used methods of Tanaka and Se-ashi [25] as developed 
in [24] to establish an extrinsic rigidity result for CHSS subject to partial vanishing of the 
first Lie algebra cohomology groups. The methods of [10] were translated to the language 
of EDS in [18]. In brief, for CHSS, if one quotients the kernel of the Spencer differential by 
admissible normalizations in the fiber one arrives naturally at the Lie algebra cohomology 
group H^{q-,q^). 

Two problems present themselves in generalizing beyond the case of CHSS. First, the Fu- 
bini systems and the EDS induced by the Lie algebra grading no longer coincide. Second, 
even if one ignores the Fubini system and restricts attention to the EDS induced by the Lie 
algebra gradings, the Spencer differential no longer coincides with the Lie algebra cohomol- 
ogy differential. We resolve the first problem for adjoint varieties by proving that integral 
manifolds of the third order Fubini system must also be integral manifolds of the (/-i, J-i) 
system. In a planned sequel, we hope to resolve this problem for general G/P. (Resolving 
the problem in individual cases is straightforward, but tedious using our current technology. 
For example, we explain how to show that the 78-dimensional variety E^/Pi C P^^'''^ is rigid 
to order five in §10.2.) The second difficulty is resolved in complete generality in this paper 
by introducing the filtered EDS (I^, O) and simultaneously examining the torsion equations 
and prolongation in lowest nontrivial homogeneous degree (see our main Theorem 6.12): 

The Spencer differential of the filtered system combined with the torsion equa- 
tions in their first nontrivial homogenous degree d is the degree d homoge- 
neous component of the Lie algebra cohomology differential . 

We hope this result will be useful in the study of other EDS with symmetry. 

Remark. The second order Fubini system induces the structure of a parabolic geometry on 
integral manifolds. When the grading on the tangent space is two-step, the third order 
Fubini system forces the parabolic geometry to be regular (e.g. one has a multi-contact 
structure on T_i). The differential invariants that measure the failure of a parabolic geom- 
etry to be locally flat take values in the Lie algebra cohomology groups H'^{q-,q) (see, e.g., 
[4, §4]). It may be interesting to compare these intrinsic and extrinsic geometries and their 
invariants in greater detail. 

1.4. Overview. In §2 we review basic definitions from exterior differential systems (EDS) 
and introduce the notion of reduced prolongations. In §3 we review the Fubini forms and 
introduce natural EDS (the Fubini systems) for studying rigidity of projective varieties. We 
then introduce the (/q, Jq) and (/_i, J_i) EDS in §4.2. In §5 we show the (I-i, J-i) system 
for adjoint varieties is implied by the third order Fubini system. 

We define the filtered EDS (/f,J]) in §6. We then review Lie algebra cohomology and 
Kostant's theory in §7.1 and §7.2, respectively. In §6 wc reduce proving rigidity for the 
(Ip, Vl) system to proving that certain Lie algebra cohomology groups are zero. In §7.3 we 
identify groups and modules for which these cohomology groups vanish. 
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We establish flexibility of Seg(F^ x P") and respectively in §8 and §9. In §9.4 we 
discuss the intrinsic geometry of the integral manifolds modeled on Z^. Finally, in §10 we 
complete the proofs of the remaining rigidity results. 

1.5. Generalizations. As discussed above, one could prove further rigidity results by hav- 
ing a better understanding of the Fubini forms. More precisely, one would need to determine 
to what order the Fubini form of a homogeneous variety must be specified in to fix the neg- 
ative part of the Maurer-Cartan form in the Lie algebra grading. We plan to address this 
question in a subsequent paper. 

All our systems have immediate generalizations to systems for homogeneous submanifolds 
G/P C G' /P' of homogeneous varieties. (That is, Pf/ may be replaced by the more general 
G' /P' .) Hong [12, 13] has results in this setting for three-step gradings. The machinery we 
have developed here will permit a more general study. 

For the study of general Schubert varieties (see, for example, [3]), or more generally G- 
subvarieties of a homogeneous variety, there is another obstruction to applying Kostant's 
theory. Kostant's method of calculating H^{xv, P) only applies if F is a g- module and n C p C 
0. It may be possible to extend Kostant's theory to apply to systems for varieties modeled 
on Schubert varieties by finding a sufficiently invariant inner product on the relevant n and 
F to enable a "Hodge type" theorem. 

1.6. Notational conventions. We work over the complex numbers throughout, all func- 
tions are holomorphic functions, manifolds complex manifolds etc... (although much of the 
theory carries over to M, with some rigidity results even carrying over to the setting). In 
particular the notion of a general point of an analytic manifold makes sense, which is a point 
off of a finite union of analytic subvarictics. We use the labeling and ordering of roots and 
weights as in [1]. For subsets X C PF, X CV denotes the corresponding cone. For a man- 
ifold X, TxX denotes its tangent space at x. For a submanifold X C PF, TxX = TpX C F, 
denotes its affine tangent space, and p G x =: L^. In particular, T^X = x*®TxX/x. We use 
the summation convention throughout: indices occurring up and down are to be summed 
over. If G is semi-simple of rank r, we write P = P/ C G for the parabolic subgroup ob- 
tained by deleting negative root spaces corresponding to roots having a nonzero coefficient 
on any of the simple roots a^^, ig G I C {1, ...,r}. 

1.7. Acknowledgments. We thank R. Bryant, B. Doubrov, J. Hwang, S. Kumar, and K. 
Yamaguchi for useful discussions. The essential step for the transition to Lie algebra coho- 
mology was resolved while A. Cap was visiting us. Remarkably Cap needed almost identical 
machinery for his work (see [5]) and, after a long day of discussions, both problems were re- 
solved independently. This paper has benefitted tremendously from our conversations with 
him. In particular, the reformulation of our results in terms of filtered EDS was suggested 
by Cap. 

2. Linear Pfaffian systems (EDS) 

A linear Pfaffian exterior differential system (EDS) on a manifold S is a flag of subbundles 
of the cotangent bundle to S, 7 C J C r*S such that the map I h?{T*Y./J) given by 
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9 ^ d9 mod J is zero. An integral manifold is an immersed submanifold i : ^ S, where 
n = dimiV = rank (J//), such that = and ?*(A"(J//)) is non-vanishing. 

A standard example is the space of one-jets S = J"'^(]R,]R) with coordinates {x,y,p) and 
/ = {dy — pdx} J = {dy — pdx,dx}. Integral manifolds are the lifts {{x, f{x), f (x))} of 
graphs of arbitrary differentiable functions / : R — > R. 

Fixing a general point a; G S we set V = {J/I)*, W = I*. Calculating dl mod / one 
defines the tableau A C W ^V* which (in the absence of torsion) essentially consists of the 
space of n-planes in Ta;S on which the forms in I vanish; equivalently, the possible first 
order Taylor series of any integral manifold (expressed as a graph) through x. Because the 
systems in this paper are modeled on homogeneous spaces, V, W and A will be independent 
of the base point a; G S: we suppress reference to it. 

Calculating dl mod / also yields the torsion, which is an element oiW0 A'^V*/6{A 0V*), 
where 6 is the Spencer differential defined below. If there are no n-planes in T-jS on which 
the forms in / and their derivatives vanish at x, one says the system has torsion at x. In 
this case one must restrict to the submanifold S' C S where there is no torsion. 

Consider / G 1^ (g) as a linear map f -.V and define 

S : {W®V*)(^V* ^ W^®AV* 

by 5(/(gi^) = A 4, and A^^^ := kei 6\a^v* ■> the prolongation of A. Cartan developed a 
test to determine the "size" of the space of local integral manifolds by comparing a crude 
estimate for the dimension of the space of admissible second order terms in the Taylor series 
(which is ^4^^)) with the actual dimension. When this test fails, one prolongs the system, 
defining a new system on S x A^^^ . 

The systems we will be dealing with all have at least one integral manifold, the corre- 
sponding homogeneous model. These systems may be grouped into three distinct categories. 

Definition 2.1. We will say a system is rigid if all integral manifolds are equivalent to the 
homogeneous m,odel. We will say a system, is quasi-rigid if som,e prolongation of its tableau 
is zero and the system is not rigid. (In other words, some prolongation of the system is a 
Frohenius system.) In this case the space of local integral manifolds is finite dimensional. 
Otherwise we will say the system is flexible. 

In the case that the system is flexible, the space of local integral manifolds is infinite 
dimensional the integral manifolds will be determined by some space of free functions, as 
in the example of (M, M) above. 

The first derived system I' C I is the maximal subbundle such that dl' = mod / holds. 
We may and will suppress it in calculations. (See [11, p. 216] or [2, p. 45].) 

A vector field X G r(TS) is a Cauchy characteristic vector field for the linear Pfaffian 
system (/, J) if a{X) = for all a G i". Cauchy characteristic vector fields correspond 
to (infinitesimal) symmetries of the EDS. By adding differential forms dual to the Cauchy 
characteristics to J, one obtains a new system whose integral manifolds are in natural 
correspondence with those of the original system. (See [11, p. 205], [2, p. 36].) 

2.1. Reduced prolongations. In our case S = GL{U), the group of invertible linear maps 
U ^ U, and / will consist of a component of the Maurer-Cartan form of GL{U). We are 
interested in submanifolds of ¥U ; the integral manifolds of our EDS will be frame bundles 
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T C GL{U) over these submanifolds. Consider the subgroup of A/"' C GL{U) that preserves 
the EDS: it will carry integral manifolds to integral manifolds. Let N d M' denote the 
subgroup that preserves the base submanifold in ¥11 . 

The prolongation coefficients in A^^^ C A®V* vary under the action of M- These 
variations are the admissible normalizations of the prolongation coefficients. The admissible 
normalizations may be realized as the image of a map 5 : n ^ A^V* ^ where n is the Lie 
algebra of N. We exploit this group action by defining a reduced prolongation of our EDS: 

^ ^(^Vlmage {5 : n ^ A®V*) . 

As far as determining integral manifolds via the Cartan algorithm is concerned, normaliz- 
ing by the action of N is of limited practical effect, but for the EDS discussed in this paper, 
the normalization corresponds to the denominator of a Lie algebra cohomology group. 

3. The Fubini systems 

3.1. The Fubini forms. Let X"- C FU = be a submanifold and let x e X. Take 
linear local coordinates on ¥U about x adapted so that X is locally graphed over its 
(embedded) tangent plane at x. Expanding the functions in Taylor series, and collect- 
ing the homogeneous terms in each degree, one obtains a scries of tensors F_j^ = F_^. x x ^ 

(x®'°"')0S''^T*X(g)Ar^.X. Here K^X = T^FU/T^X denotes the normal space. If the lo- 
cal linear coordinates are {x^, ...,x'\x^''^^, ...,x^) = {x'^^x^) so that X is locally given by 
equations x^ = f'^{x°') then 

FkXx = (-1)'' ^ ■ dx'' o . . . o dx"' O 

The F_^. depend on the choice of linear coordinates. To rectify this ambiguity, let tt : J-^ X 
denote the bundle of first order adapted frames, that is 

.F^ = {(eo,...,ejv) e GL{U) \ [eo] e X, f[,„]X = span {cq, e„}} C GL{U) . 

Here the ca are to be considered as column vectors. The fiber over x is isomorphic to the 
space of first order adapted linear coordinates based at x. Thus one obtains well defined 

Fk G T{T^,TX*{L^-^ g) S^T*X (g) NX)) which are called the Fubini forms [11]. Here L is the 
line bundle 0^jj{—^)^ the bundle with fiber Lx = x. We use the notation := L®^. The 
Fubini forms determine X up to projective equivalence. 

3.2. The Maurer- Cartan form. Let G be a Lie group, let oo G Q.^{G,q) denote the 
Maurer- Cartan form of G. By definition u is the unique left- invariant differential form such 
that LOid : Tj^G ~ g — > g is the identity map. It satisfies the Maurer-Cartan equation 

(3.1) du = — [u, u] . 

To be precise, given two g-valued 1-forms p and rj, and two tangent vectors u, v, we define 

[p,ri]{u,v) := [p{u),ri{v)]-[p{v),ri{u)] . 

In practice, p and rj will be components (of a fixed homogeneous degree) of the Maurer- 
Cartan form. 

If G C GL(U) is a matrix Lie group, the Maurer-Cartan equation may be written dui = 
—Lo A cj: if o; = (w^), with a;^ the entries of the matrix, we have {dLo)^ = —uIq A cj'^. 



FUBINI-GRIFFITHS-HARRIS RIGIDITY AND LIE ALGEBRA COHOMOLOGY 



9 



An essential feature of the Maurer-Cartan equation is that differentiation is reduced to an 
algebraic operation. 

3.3. Second order Fubini systems. Fix vector spaces L,T,N of dimensions l,n,a and 
fix an element F2 G S'^T* (E) N (E) L. Let U = L®TeN, and let co € n^{GL{U),gi{U)) 
denote the Maurer-Cartan form. Write gl(U) = {L®T ® N)* <^{L®T ® N) and let, for 
example, ojl*®t denote the component of uj taking values in L* ® T. We have 

(^L* ® L i^T* ® L 
^L* ® T ^T* ® T 
i-^L* ® N ^^T* ® N 

Writing the Maurer-Cartan equation component-wise yields, for example, 

ck^L*i^T = —^L*®T A Ul*^L — OJt*®T A Ul*®T — ^N*®T A OJL*t^N- 

Given F2 G L (g) S'^T* (g) N, the second order Fubini system for F2 is 

-^Fub2 = Wl*®N, i^T*®N - F2{oJl* ®t)}, J¥nh2 = {-^Fub2i ^L*®t}- 

Its integral manifolds are submanifolds C GL{U) that are adapted frame bundles of 
submanifolds X C ¥U having the property that at a point x E X, the projective second 
fundamental form F2^x,x is equivalent to F2. (The tautological system for frame bundles 
J^^ of arbitrary n dimensional submanifolds is given by / = {ll>l*(^]\[}, J = {/, i^l*®t}-) 
Let R C GL{L) x GL{T) x GL{N) denote the subgroup stabihzing F2 and let 

r C (L* (g) L) © (T* ® T) e {N* ® N) =: fll(?7)o,* 

denote its subalgebra. These are the elements of 0l(f/)o,* annihilating F2. (The motivation 
for the notation 0l(J7)o,* is explained in §4.1.) Since r is reductive, we may decompose 
0t(^^)o,* = tSt"*" as an r-module. In the notation of §2, 

V ~ L*(^T, W ~ (L*®iV)©(T*(8)iV), A , 

with L* (g) AT in the first derived system. That C V* ®W may be seen as follows 

d{u!T* ^ N — F2{u!l* ^t)) = — ^T* ® L ^ ^^L* ® N — i^T* ®T ^ ^^T* ® N 

+ -^2 ('^L* ® T A Wt* igi T + ^L* ® N f\ ^N* ®t) 

(3.3) = — wt* A F2(a;L* ®t) — -^2((^L* ®t) A wat* ® at 

— F2(— cJi* 0i A wl* (g)T — ^l*®t^^t*®t) mod/ 
= {ujQ^^ . F2) ^UJL* ®T mod/ 
= {uj^^ . F2) ^UJL* ®T mod/. 

Above, ti;o,* • ^2 denotes the action of the g[(C/)o,*^valued component o^q,* of the Maurer- 
Cartan form on F2 € S'^T* N . Recall that r is the annihilator of this action. By definition 

<^0,* • F2 = {lOx + UJ^±) . F2 = L0^± . F2. 



iON* (^N 
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3.4. Third order Fubini systems. Fix F2 G L(g,S^T*(g)N and F3 G L'^ (gi S^T* (g, N . 
The third order Fubini system corresponding to (^2,^3) is 

(3 4) ^^^^ ^ {^L*(8N , <^T* ® AT - -PaC^^L* ®t) , UJ^-L-F2 " -P3(^L* (^t)} 

where the last term in /pubs is iV T"* (g) T* (8)L- valued, and uj^±.F2 is as in §3.3. Here 

V ~ L*0r, w ~ (L*(8)iV)e(r*® Ar)er^, 

with the first two terms of W in the first derived system. The tableau A (L 0T*) ® {T ^ N*) 
sits in y* by a calculation similar to (3.3). 

Integral manifolds of the third order Fubini system are adapted frame bundles JF^ C 
GL{U) over submanifolds of projective space whose second fundamental form is isomorphic 
to F2 at each point and whose Fubini cubic is normalizable to F3 at each point. Note that 
these systems admit reduced prolongations. The reductions (or normalizations) are by the 
subalgebra n C (L (g) T*) © (T O A^*) © (L N*) such that n.(r^. - F^{L* ® T)) = 0. 

In this paper we will be concerned with (^2,^3) modeled on the Fubini forms of a homo- 
geneous variety. 



4. A BI-GRADING OF Qi{U) 

4.1. The osculating filtration. Given a submanifold X C fU, and x E X, the osculating 
filtration at x 

Uo cUi G ■■■ GUi = U 

is defined by 

Uo = x, Ui = f^X, U2 = Ui+ F2{L* S^T^X), Ur = Ui_i + Ft{L*^-^ (g) S^F^X). 

The Fj arc the Fubini forms of §3.1. We may reduce the frame bundle J-]^ to framings 
adapted to the osculating sequence by specifying e = (eg , , e^^ , . . . , e^^ ) G J^x so that 
Uk = span {eo, 60;, > • • • > Sju^. }. (The indices a and /x^ respectively range over 1, . . . , n and 
dimC/fc_i + 1, . . . ,dim?7fe.) From now on we work on this reduced frame-bundle, denoted 

At each point of obtain a splitting of U. This induces a splitting 

Q{{U) = ffi0[(f/)fe,* . 

The asterisk above is a place holder for a second splitting given (in §4.2) by the represen- 
tation theory when X = G/P. 

The osculating filtration of U determines a refinement of the Fubini forms. Let iV^, = 
Uk/Uk-i and decompose Fg = ® kFk,s so that Fk^s '■ L*^~^ ® S'^T^X N^. Although the 
Fubini forms do not descend to well-defined tensors on X, the fundamental forms F^^j^ do. 
By definition, Fk^k ■ L*^~^ (g) S^T^X — N^^xX is surjective. 

Remark. The only nonzero Fubini forms of a homogeneously embedded CHSS are the fun- 
damental forms. For the adjoint varieties, the only nonzero Fubini forms are ^2,2, -^2,3, -^2,4- 
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4.2. The root grading and the {Ip, Jp) systems. Let g be a complex semi-simple Lie 
algebra with a fixed set of simple roots {ai, ar}, and corresponding fundamental weights 
{uJi, . . . ,iOr}. Let / C {1, . . . ,r}, and consider the irreducible representation fi : q ^ flK^) 
of highest weight A = X^jg/A^a;^. Set g = ij,{q), and let //(G) C GL(U) be the associated 
Lie group so that G/P C ¥U is the orbit of a highest weight line. Write P = P/ C G for 
the parabolic subgroup obtained by deleting negative root spaces corresponding to roots 
having a nonzero coefficient on any of the simple roots ai, i € I. 

Since g is reductive, we have a splitting sl{U) = flffifl"*", where is the g-submodule 
of qI{U) complementary to g. Let u G Q^{GL{U),qI{U)) denote the Maurer-Cartan form 
of GL{U), and let Ug and ujg± denote the components of u taking values in g and g-*-, 
respectively. 

The bundle J^Q/p admits a reduction to a bundle = IJ-iG). On this bundle the 

Maurer-Cartan form pulls-back to take values in g, that is, u!g± = 0. Conversely, all dim (G) 
dimensional integral manifolds of the system / = {<-t^gx} arc conjugates of That is, the 

ujg± = system is rigid. Theorem 6.12 establishes the rigidity of weaker systems, subject 
to the partial vanishing of components of a Lie algebra cohomology group. 

Let Z = Zj C the the grading element corresponding to J2iei ^i- '^'^^ grading element 
Zi for a simple root ai has the property that Zi{aj) = Sj. In general Z = X^ie/ ^i- Thus, 
if (c~^) denotes the inverse of the Cartan matrix, then given a weight i' = ^ u^uij, 

(4.1) Z{v)= J2 

l<j<r 

iei 

The grading element induces a Z-grading of g = (B'Lj^Qk- To determine k in the case P = Pa^ 
is a maximal parabolic, let a denote the highest root. Given a = we have k = rrii. 

The module U inherits a Z-grading 

u = Uz(x) e Uz(x)-i e • • • © Uz(x)-f . 

The Uj are eigen-spaces for Z. This grading is compatible with the action of g: ^{Qij.Uj C 
Ui+j. We adopt the notational convention of shifting the grading on U to begin at zero. 
The component Uq (formally named Uz(x)) is one dimensional, corresponding to the highest 
weight line of U, and G-PC/q = G/P C ¥U. (The labeling of the grading on gi{U) = U*(g)U 
is independent of our shift convention.) 

Note, in particular, that the vector space T^i^j{G / P) /Id — g/p is graded from —1 to —k. 
The osculating grading on U induces gradings of qI{U), g and g-*-. 

We write 

where the first index refers to the osculating grading (§4.1) induced by G/P C FU and the 
second the root grading. We adopt the notational convention 



s 
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SO if there is only one index, it refers to the root grading. Note that although Z[X) need 
not be an integer, / is. So the grading of Qi{U) is indexed by integers — /, /. Moreover, 
this grading of Qi{U) is independent of shift convention. 

We began with frame bundles J- C GL([7). But rescaling cq we may assume that T C 
SL{U), and wc will do so from now on. We define the {Ip, Jp) system on SL{U) by Ip = 
{WgX }, Jp = {/p,ajg }. The EDS we will study are the filtered systems which are 

weaker than the (Ip, Jp) systems. (See §6.) 

Remark. The osculating grading coincides (up to a change of sign) with the Lie algebra 
grading if and only if G/P is CHSS. 

5. The adjoint varieties 

In this section we describe the fundamental adjoint representations from a uniform per- 
spective and prove that integral manifolds of the third order Fubini system for them are 
automatically integral manifolds for the corresponding (/_i, J_i)-system. 

Remark. We have the following geometric models for adjoint varieties Z^^. 

o An = SLn+iC: The fiag variety Fi,„(C"+i) of lines in hyperplanes in C"+^ 
o Cn = Sp2nC: The Veronese variety z;2(p2"-^) C P(5'2C2"). 

o G = SO{n): The variety Gq{2, n) of 2-planes in C" on which the quadratic form restricts 
to be zero. 

o G = G-2. The variety Gnuii(2, lm(0)) of 2-planes in the imaginary complexified octonions 
to which the octonionic multiplication restricts to be identically zero. 

o G = Eq: The variety of P^'s on the Cayley plane. 

The highest root a induces a five-step grading q = 0_2 ® 0-i © flo ® 01 © 02- (The corre- 
sponding grading of [/ = is Uj = Qj-2-) The adjoint variety is the G— orbit of P02. 

Fix a Clicvallcy basis of 0. Decompose 0o = f + C{Za} into the semi-simple Levi factor f, 
and the one-dimensional (assuming 7^ On) C{Za} ■ In order to obtain the representation 
= ad(0) C 0[(^7) we make the following observations. First, Z^ acts on qj by jld; and f 
acts trivially on 0-|-2- Let : f 0[(0_i) = 0_i ® denote the action of f on 0_i. (Our 
choice of notation is made to be compatible with the geometry that enters later.) The Killing 
form B on allows us to identify Q±i ~ 0zpi*, so that we may regard //~^ as an clement 
of f* (8)0-1 (8 01- The bracket [•, •] : 0_i x 0_i 0_2 induces a symplectic form ft € A^0l^. 
The linear map 0_i —>■ 0^^ = 0i induced by CI is an f-module isomorphism. (That is, the 

representation is symplectic.) Let 11"^ = contr{B (8) 11"^ (8 ^i) £ 0!.i (8) 0li <8) f denote 
the natural contraction 

(f <8) f) (8) (f ® 0I1 (8) 0-1) (8) (0I1 0I1) ^ f 0^1 (8 0I1 . 

Because the representation is symplectic, the image is in f (8-5^011. 

The canonical identification Q±i ~ 0^^ allows us to treat 11"^, 11"^ and Q, as elements 
of f (8) 5*^01, f* '8)01 '8)01 and A'^qi, respectively. 

The adjoint variety admits a frame bundle fi{G) = J^^^ C GL{U) on which the Maurer- 
Cartan form pulls-back to take values in C 01(^7). Decomposed with respect to the 
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bi-grading, <jO takes the form 

0-1 

0-2 




V 



(5.1) 



01^ 

Zald + //-2(f) 

-^f^(0-i) 
//-'(0-i) 
-0-2 Id 




02 

J^(01) 




-0-1 

-20-2 






^ ad(f) 

//-'(0-i) 




-5 02 Id 



^ 


"2 02 




-//-^(qsi)) 

-Zald + //-2(f) J)(0i 



The last non-zero fundamental form is the second, so the osculating filtration has length 
two. The tangent space T = Ni and the normal space N = N2 decompose as A'^i,-! © Ni^_2 
and A'^2,-2 © ^2,-3 © -^2,-4, respectively, under the root grading. We write Tj = Nij and 
Nj = N2J. Notice that is the restriction to iV*2 of II : N* ^ S^T*. The induced 
bi-grading on fl[(i7)osc,aig is indicated in the table below. 





L* 




T_2* 


iV-2* 




iV_4* 


L 


(0,0) 


(-1,1) 


(-1,2) 


(-2,2) 


(-2,3) 


(-2,4) 




(1,-1) 


(0,0) 


(0,1) 


(-1,1) 


(-1,2) 


(-1,3) 


r-2 


(1,-2) 


(0,-1) 


(0,0) 


(-1,0) 


(-1,1) 


(-1,2) 


iV_2 


(2,-2) 


(1,-1) 


(1,0) 


(0,0) 


(0,1) 


(0,2) 


iV_3 


(2,-3) 


(1,-2) 


(1,-1) 


(0,-1) 


(0,0) 


(0,1) 


N_4 


(2,-4) 


(1,-3) 


(1,-2) 


(0,-2) 


(0,-1) 


(0,0) 



Proposition 5.2. Every integral manifold of the third-order Fubini system (/pubs , -^Fubs ) 
for a given adjoint variety is an integral manifold of the (/_i, J_i) system for the same 
adjoint variety. 

Proof. Suppose that !F C SL{U) is an integral manifold of the third-order Fubini system. 
We wish to show that the 0;^^g-valued component of the Maurer-Cartan form vanishes when 
pulled-back to JT. That the g^Q ^-valued component vanishes is an immediate consequence 
of the injectivity of the second fundamental form F2 on each homogeneous component. 

Referring to the table above, we see that there remain four blocks of the component of the 
Maurer-Cartan form in q^.^q to consider: the three (0, —1) blocks ljt_2^tZi , (gi Ar*^ and 
i^N-i^Nta'j ^'^d the singleton (0,-2) block ojn-4^nZ2- "^^^ third Fubini form is defined 
by (3.5) of [11, §3.5]. The vanishing of the g-'--component of the first two blocks is a 
consequence of the S'^T*i^N-3 component of F3. (This is the only nonzero component 
of F3.) The vanishing of the g^-component of the third and fourth blocks is given by the 
53 (T* ) (g) Af_4-component of F3 . □ 



6. Filtered EDS 

In [10], it was observed (in different language) that for rigidity problems associated to 
CHSS, the (/-I, J-i) system could be proved to be rigid using Lie algebra cohomology. More 
precisely, the Spencer differential coincides with the Lie algebra cohomology differential dl 
and the admissible normalizations coincide with the image of the Lie algebra cohomology 
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map di- This correspondence breaks down when A; > 1 (see Remark 6.7 below), but it can 
be restored with the use of filtered EDS and simultaneously studying the prolongation and 
torsion. 

Definition 6.1. Let Tj he a manifold equipped with a filtration of its tangent bundle C 
C • • • C T^f = TS. Define an r-filtered Pfaffian EDS on T, to be a filtered ideal 
I C T*T, whose integral manifolds are the immersed submanifolds i : M —>■ T, such that 
i* {Iu)\i* (Tu-r-^ = for all u, with the convention that T"^ = TE when —s < —f. 

Another way to view filtered EDS is to consider an ordinary EDS on the total space of 
the sum of the bundles (rS/r"+''). In our case these bundles will be trivial with fixed 

vector spaces as models. 

Define {ll, to be the {p+ l)-filtered EDS on GL{U) with filtered ideal P := to ± and 

independence condition Q given by the wedge product of the forms in LOg_ . We may view 
this as an ordinary EDS on 

GL{u) X ([B^(g)(0-2e ■■■ ©0-fc)*] e [s^_i 0(5-3 ■■■ eg-fc)*]e---e[0^_fc+2®0-fc]) 

where, giving 0^ (g) 0^^- linear coordinates \ij , we have 

(6.2) /J = {u;g±, s<p-k + l; ujg±_^^^- Xp_k+2,k{^B-k) ^ 

^3p-k+3 ~ ^P-k+3,k{^0-k) - Ap-fe+3,fc-l('^fl_fc+i) , ■ ■ ■ 
•^fl^ ~ ^P,k{^0-k) Ap,2(<^fl-2) } 



We henceforth assume p > — 1. 

6.1. The p = —1, k = 2 case. We give a proof of the main result in §6.2 below, but for 
the reader's convienence we work out the p = —1 and k = 2 case explicitly to illustrate the 
central ideas. 
Abbreviate 

Notice that 0^ = sl(f/)s for all s < —3. so that u>g± = ojg, for all s < —3. Thus the ideal 
III is 

ill = {'^0^^-^-1,2(^0-2), ^^0^2' ^-3' ••• ' ^-f} ■ 

The calculations that follow utilize the Maurer-Cartan equation (3.1), and the facts that 
[05 0] C and [0,0"*"] C 0-*-. It is easy to see that dug = modulo lli when s < —4. Next, 
computing modulo ll^, 



(6.3) 


-du>-3 = 










-2, A-l,2('^£ 


- 


2)] , 






(6.4)- 


-dco ± = 

0-2 




2 '^00^. 


+ 


^0- 


-l'^0^1. 


+ 






'^0-1. 


0^ 






UJg_ 


2 ''^00^. 


+ 




1, A-1,2 


('^0- 


.2)] + 


[A-i,2(ws-2),A_i,2(a;0_2)]gX 
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Finally, 
(6.5) 

~^ ('^0^1 ~'^-1.2('^fl-2)) = 



+ 



+ 



^0^1 '^00 



+ 



+ dA_ 



1,2 



+ 



'^0^1 'Six 



00 



+ [A_i,2(a;0_2),a;go] 



+ 



A-i,2(wg_2),a;0xJ^^ + dA_i,2(Aa;g_2) 

- A_i,2 ([u;g_2,a;go] + [a;0^,a;g_i] + [A_i,2(a;g_2), A_i,2(a;0_2)] J 

Here the bracket is as indicated in §3.2 and [•, -Jg (resp. [•, Jgi) denotes the component of 
the bracket taking values in g (resp. g-*-). 

The three differentials above must vanish on an integral element modulo Notice that 
the vanishing of (6.4) implies 



00 



Ao,l('^0-i) + Ao,2('^0_ 



for some Aqj G 0o glj- 

Now consider the degree one (1 = p + 2) homogeneous component of (6.3,6.4,6.5), si- 
multaneously examining the torsion and tableau. In order the have an integral element, 
Ai := ®s=_iAs,i_s must be in the kernel of the map 



5i ■ 



i(9^®0-.-i) ^ (0-i<8)0li Agli)e(0i2«'Sli Agla) 



defined as follows. Given u-\,v-i G g_i, 

(6.6) (5i(Ai)('u_i Av-i) = [Ao,i(u_i),w_i] + [n_i, Ao,i('y-i)] - A_i,2([u_i, v_i]). 
For u-i G g-i,v_2 G g_2 

(5i(Ai)(n_i A'u_2) = [Ao,i('U-i), v-2] + [-u-i, A_i,2(?^-2)] • 
That \s, 5\ = d\, where d\ is the Lie algebra cohomology differential described in §7. 

Remark 6.7. Note that had we instead used the (7_i, J_i) system in equation (6.6), the 

A-1,2 term would be missing and we would not have The map 61 is akin to an 'augmented 
Spencer differential' it addresses both the torsion and prolongation in homogeneous degree 
1. 

Moreover, g^ = nng[(?7)i (cf. §2.1), and the Lie algebra cohomology denominator di{'Qi) 
is the space of admissible normalizations of the prolongation coefficients Ai. Thus, the 
vanishing of H\[q-.,q^) implies that normalized integral manifolds of the system 
are in one to one correspondence with integral manifolds of the (/q , il) system. 



6.2. The general case. Define 



A., 



® As 

j=p+2-s 



2i 

j=p+2-s 
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and set 

k 

^s,* (<^0<o) '■= X] ^s,j i^s-j) ■ 

j=p+2-s 

Defining Xgj = when s + j < p + 2 allows us to write the generators of the ideal 1^ 
compactly as Ug± — A^^^, {'^s<o ) • 

Observe that, if s < —k, then = sl{U)s, so that Ug = modulo ll. Also, w ± = 
modulo Ip. Now, the Maurer-Cartan equation (3.1) yields, modulo Ip, 



, wt']g± + 2 dAs^* A Wg^Q + A, 

t+t'=s 
s—p—1 



-k<t+t'<-l 



Ss-t 



t=-k 



(6i 



t=s—p 

s+k 

fl<0/ 

t=p+l 

k / -j-p-l 

- E ^.4 

j=p+2-s 



+ 2 dAs * A a; 



t=-k 

P 

+ E ['^S* + ^t,*K<o)> '^fl-,-t + '^-j-t,*K<o)] , 
t=-j-p 
k-j 

t=p+l 



S<OJ 



Observe that, for s < —2k, dugi. = modulo 7^; that is, for s < —2k, LOg± lies in the first 
derived system I' . 

The necessary vanishing of (6.8) on integral elements tells us that there exist Xsj, taking 
values in gj- ^ q^j*, such that 



(6.9) 



CO 



- ^A^j((j0_.) , p+l<s<p + k. 



The vanishing of (6.8) also imposes consraints on the uj^^ and dA^,*, but these terms will not 
appear when we consider the first non-zero component of the augmented Spencer differential 
6p^2 below. (Of course, the Wg^, s > 0, are dual to the Cauchy characteristics, and thus 
ought not influence the computation.) 
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Careful consideration of (6.8) shows that the right-hand side contains no components of 
homogeneous degree less than p + 2. We identify the necessary condition that the degree 

p + 2 homogeneous component of the right-hand side of (6.8) vanish on an integral element 
by evaluating the expression on u £ j^-i^j'i-t ^ q_^^ y ^ j'-iyj-i-i ~ g_^. -v^rith s = 
{p + 2) — {i + j), I < i,j < k. Utilizing (6.9), we have 

-2d(a;0j_ - A^,*(a;0<J)(n, u) 

s—p—l 

= Yl K*(^)'^5-t,*K<o(^))] 

t=-k 
P 

t=s—p 
s+k 

+ Yl [VK<o(«))''^S.-t(^)] 
t=p+l 

- As,* {[uj{u),uj{v)]) 

[u;g{u),\s-i,4ujQ^^iv))] - [u;g(z;),As-j>((j0<„(n))] - Xs,*i[LVgiu),ujg{v)]) . 
In particular, (Ap+i,i, Xp^2, ■ ■ ■ , Xp+2-k,k) must lie in the kernal of the map 

defined as follows: given x G g_i and y G Q-j, 

(6 10) "^P+^C-^P+l,!' V2)---)'^p+2-fe,A;)(a;,i/) = 

[x,\p+2-j,j{y)] - [y,\p+2-i,i{x)] - Xp+2-i-j,i+j{[x,y]) . 

As WG will see below, 5p^2 = dp+2 degree p+2 component of the first Lie algebra 

cohomology differential (described in §7.1). As in §6.1, (5p+2 should be viewed as the Spencer 
differential augmented to deal simultaneously with tableau and torsion. 

Moreover, 0^_^2 = H gl(J7)p+2 (cf. §2.1), and the Lie algebra cohomology denomina- 
tor 5p+2(Sp+2) is space of admissible normalizations of the prolongation coefficients 
Xp+2-m.rn- Thus, the vanishing of H^_^_2{Q-,Q'^) implies that normalized integral mani- 
folds of the (/p, 0) system are in one to one correspondence with integral manifolds of the 
{Ip_^_l,Q,) system. We have established the following lemma. 

Lemma 6.11. Let U be a complex vector space, and q C 9l{U) a represented complex semi- 
simple Lie algebra. Let G/P C PC/ be the corresponding homogeneous variety. Denote the 
induced Z-gradings by g = Q-k © • • • © 0fe and U = Uq® ■ ■ ■ (B U-f . Fix an integer p> —1, 
and let {Ip,fi) denote the filtered linear Pfaffian system given by (6.2). 

/f ffp_l_2(0-, B"*") = 0, then normalized integral manifolds of the Ip system are in one to 
one correspondence with integral manifolds of the Ip^i system. 

Now a straightforward induction yields our main result. 

Theorem 6.12. Let U be a complex vector space, and g C Qi{U) a represented complex 
semi-simple Lie algebra. Let Z = G/P C PC/ be the corresponding homogeneous variety 
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(the orbit of a highest weight line). Denote the induced Z-gradings by q = Q-k © • • • ©0fc 
and U = Uq® ■ ■ ■ (B U-f. Fix an integer p > —1, and let {Ip, J7) denote the linear Pfaffian 
system given by (6.2). If H^{q-,q-^) = 0, for all d > p + 2, then the homogenous variety 
G/P is rigid for the (Ip,^) system. 

7. The (/^, r2)-SYSTEMS and Lie algebra cohomology 

7.1. Lie algebra cohomology. Let [ be a Lie algebra and let F be an [-module. The maps 

: AH* ^ A^+^l* 

are defined in a natural way to respect the Leibnitz rule [14], and give rise to a complex. 
Define ^^'^(l, F) := ker d'^ /Image d^~^ . We will only have need of and which are as 
follows. If X G F and v,w E i, then 

d'^{X){v) =v.X, 

and if a (8) X G AH* (8) F, then 

(7.1) d^{a0X){vA w) = a{[v, w])X + a{v)w.X — a{w)v.X . 

Now let i be a graded Lie algebra and F a graded l-module. The chain complex and Lie 
algebra cohomology groups inherit gradings as well. Explicitly, 

: ©i(U)* OFrf_i ^ (Bj<m (l-i)* A H-„,)* ^Td-j-m ■ 

Taking [ = 0_, F = g-*- and d = p + 2, we obtain 5^+2 = "9p+2 ™ system as asserted 

following (6.10). Moreover, the image of 9p_|_2 is the space of admissible normalizations. 

7.2. Applying Kostant's theory. Kostant [14] shows that under the following circum- 
stances one can compute i?*^(l, F) combinatorially: 

(1) [ = n C p C g is the nilpotent subalgebra of a parabolic subalgebra of a semi-simple 
Lie algebra q. 

(2) F is a g-module. 

Under these conditions, letting go C p be the the (reductive) Levi factor of p, H'^{n,r) 
is naturally a 0o-™odule. When F is an irreducible g-module of highest weight A, the 
irreducible go-modules appearing in H^{n,T) have highest weight w.{X), w G yV^{k). Here, 
W^{k) is a subset of the Weyl group of g, and w.{X) = w{X + p) — p denotes the afiine action 
of w, with p = iOi = ^ Ylii (^i the sum of the fundamental weights of g. 

For the 1^ system associated to G/Pi we will only be concerned with i!f^(g_, g-*"). In 
this case yV''(l) is the set of simple reflections ai corresponding to simple roots Ui such 
that i € I. We will use the grading element Z, introduced in §4.2, to decompose the first 
cohomology group into homogeneous components H^{Q-,g-^) = iJ^(0_, g-*-). 

Remark. Our g_ is Kostant's n* = [*. In particular, if^(g_,F) = ifi^(n, F); and it is the 
latter that we shall be computing via Kostant. 
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7.3. Computing iif^(g_, 0"*"). By §7.2, we need only calculate the affine reflections in the 
simple roots and to determine if they are nonzero in degree greater than p+ 1. We follow 
[10, 24] in this subsection. 

Let £7^0 denote the affine reflection through simple root aig, zq G /. If we apply a^^ to 
A = X^Ui we obtain, from Equation (4.1), 

i,s 

To see this, let (-, •) : t* x t* ^ C be the pairing determined by (wj, aj) = Sij. 
Z{a^,^.X) = Z{aa,^{X + p))-Z{p) 

= Z{X + p- {X + p,aio)aio) - Z{p) 

= Z{X) + Z{p) - (A + p, ai,)Z{aiJ - Z{p) 

= Z{X) - {X + p,aio) 

= ^V(c-^)m.-(A«o + 1) 

i,s 

In particular, for the trivial representation, Z[a.O) = — 1. 

For positive weights A, Z{aa^^-X) > —1 when (c~^)ig^iQ > 1; and Z{aaiQ-X) > —1 when 

{c~^)io,io > 1- R-ecall, our iJ](0_,r) is Kostant's H^^{n,T) (cf. §7.2). Thus, 

H^{q-,T) = 0, for d> 1 (resp. d > 1) when (c~^)i(,,io > 1 (resp. > 1), for all io G /. 

The inverse Cartan matrices satisfy {c~^)jj > 1, except for the following cases (omitting 
redundancies): 

(c~^)l,l = (c~^)n,n = for An 

(c"^)2,2 = 1 for ^3 

(c~^)i,i = 1 for Bn, n>2, and n>4. 

By Schur's lemma, if g is a semi-simple Lie algebra and V is an irreducible 0-module, then 
the trivial representation occurs exactly once in 0l(l^) = V* and not at all in 5l(y). 
These observations, coupled with Theorem 6.12, allow us to deduce Theorem 1.7 and 

Theorem 7.2. Let G be a complex semi-simple group and let G/P C Pf/ he a homoge- 
neously embedded homogeneous variety. Assume that G/P contains no factor corresponding 
a homogeneous variety A^/Pi, with lornin I. Then, the (Iq, O) system on SL{U) is rigid. 

Proof of Theorem 1.2. By Proposition 5.2 integral manifolds for the third-order Fubini 
system for adjoint varieties are also integral manifolds of the (/-i, J-i) system and hence 
the (7^2 system. Theorem 1.2 now follows from Theorem 1.7 and Lemma 7.3 below. 

□ 

Lemma 7.3. The (1:^^,0) system is rigid for U = U^"_^^^, n > 2. The {Iq,^) system is 
rigid for U = U^^^^^ . 

Remark. This establishes Theorem 1.8. 
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Proof. We have the decomposition, 

SO 

9 = f^2(a;i+a;„) ® ^2a;i+a;„_i ® f4;2+2a;„ ® t4;2+a^„-i ® f4;i+a;„ • 

We calculate, assuming n > 2 (note that when n = 2, uJn-i = 

Z{crai^-2{uji+ujn)) = 1, io = l,n 



■^(o-ai-(<^2 + 2Un)) 



1 n > 3 
-1 n = 2 

2 n > 3 
2 n = 2 



^((7"ai-('^2 +a;n-l)) = 
Z{(yai^-{uJi+UJn)) = 0, io = l,n- 



1 n > 3 
n = 2 



The Z{aa^.Xys may be determined by symmetry. □ 
Now we consider some examples of the {Iq, system. 

Example: Fuhini's theorem. The third order Fubini system for a quadric hypersurface is 
equivalent to the (/q, VL) system once one observes Ann (//) = flo + C C qI{U)o. We obtain 
a calculation free proof of Fubini's theorem. 

Rigidity of the (/q, fi) system can only fail for An/Pj if 0^ contains a module with large 
uJi (resp. Un) coefficient and 1 G J (resp. n E I), and all other Uj coefficients are relatively 
small. Theorem 1.9 is an example for which rigidity holds. 

Proof of Theorem 1.9. For Wd(P") C PS"^C"+^ = FU we consider U(giU* = Ud^^ (g) Udu>„ = 
Y^i=o^ii'^i+iOn)- Hence = Ui(^i+^r.)- We calculate Z{aai.i{ui + = i-l. 

A similar argument establishes the second half of the theorem for U^^_^_,^^. (The rigidity 
of U^^\^^, n > 2, is given by Lemma 7.3.) □ 

We show in §9 that the nontrivial Lie algebra cohomology group H\ that occurs from 
Z{aa-^.{^\)) (here Un-i = uji) indeed makes the (/-i, J-i) system flexible. 

8. Flexibility of Seg{F'^ x P") 

Here we prove Theorem 1.4. Fix vector spaces E and F of dimensions 2 and n + 1, 
respectively. Let Seg(FE x FF) C F{E(SiF) denote the Segre variety. If n = 1, then 
Seg{F^ X P^) is a quadric hypersurface; quadrics are flexible at order two and rigid at order 
three by Fubini's theorem [8]. So we assume for the rest of this section that n > 1. We 
have already seen above that Seg(F^ x P") is rigid to order three because it is rigid for the 
(/o, Jo) system. 

We consider the (I-i, J-i) system which agrees the second order Fubini system in this 
case. Here W = g^^, A = Qq, and V = g^i. 

However, i^|(g_, g-*-) ~ C", which as a go = C ® (sl„ + C) module is acted on by weights 
—4 for the first C C s[2 and 1 for the second C C On and is the dual of the standard 
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representation for the sl„ factor, corresponding to the weight —Arj + uj^ where rj is the 
fundamental weight of a\ and ujj are fundamental weights of On- 

We now take our EDS to be the reduced prolongation. At this point the Spencer dif- 
ferential still coincides with the Lie algebra cohomology group, but the tableau no longer 
corresponds to a g-module, so we do not see any obvious way to use Kostant's theory. Fortu- 
nately in this case the calculation is straightforward and the prolonged system is involutive, 
with characters (si, S2, s^+i) = (n^ + 2n, 0, ...,0). Moreover, it is not difficult to check 
that any Y C P^^+i agreeing with Seg to second order admits a (reduced) frame bundle 

C GL2n+2C on which the Maurer-Cartan form pulls-back to 

\ 









1 .1 


, ,1 





^0 













,0 



13 



1)1 



Here 2 < a < n + 1, and a = a + n. 



and forms uJq, 



UJU UJ„, LOn, UJ, 



0' ^0 ' ^1 



and LO^ are linearly 



independent. The rf terms were introduced in the first reduced prolongation, and the 
and coefficients in the second. 

Given an integral manifold of the system, let Y C p2"+i denote the corresponding base 
manifold. By construction, Y agrees with Seg{¥E x FF) to second order. Third order 
agreement holds if and only if the Fubini cubic vanishes on Y. This is equivalent to rf = 0. 

This completes the proof of Theorem 1.4. 

Remark. All (base) integral manifolds Y of our system are ruled by P'*s. Consider the 
induced curve in the Grassmannian G(P"', P^"+^). Such curves agree with the curve induced 
by Seg(¥E x ¥F) to order two, but differ with this curve at order three. In a remark in 
the introduction in [17] it was mistakenly stated that the above system was rigid, the error 
was primarily caused by the misconception that a curve in the Grassmannian would be 
determined by its second fundamental form. 



9. The (/_i, J_i) system for A2 

Here we consider Y^ C P^ with frame bundle J-^q on which the s[(a2)<o component of 
the Maurer-Cartan form lo agrees with that of the s[(3, C) adjoint variety X. Explicitly, 



(9.1) 



UJ 
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, 1O 
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The = (3ai are not assignments, merely renamings. 
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The quadrics of the second fundamental form are 

(9.2) = -2aia2 = -la^a^ 

(f' = -2 0203 = -2a3^. 

The tableau is torsion-free with characters (51,52,53) = (17,4,0). In the course of pro- 
longing, the 1-forms and iof are forced to be semi-basic (linear combinations of the a"). 
There exist rcscalings of the ej, e = (cq, . . . , 67) G J-, that preserve the EDS and scale the 
a^-coefficient of co^ and the a ^-coefficient of tof to be constant. We decompose the analysis 
into three cases: that the constants are {0,0}, {0, 1} (two symmetric cases) or {1, 1}. 

9.1. Case 1. Assume that both constants are zero. Taking the second prolongation shows 
that Lo satisfies the (/q, Jo) system. Thus, integral manifolds of (9.1) are frame bundles over 
the adjoint variety Z^^. 

9.2. Case 2. Suppose that = mod a^,a^; and u>f = mod 0^,0:^. After re- 
stricting our parameter space (twice) to remove torsion, the second tableau has charac- 
ters (51,52,53) = (13,1,0). The (reduced) second prolongation is of dimension 9. Again, 
we must restrict the parameter space (twice, again) to obtain a torsion-free third tableau 
with characters (51,52,53) = (9, 1,0). The third prolongation has dimension 5. The fourth 
tableau (after one reduction for torsion) has characters (51,52,53) = (4,0,0) and is involu- 
tive. The Maurer-Cartan form (the components of non-negative degree) is given below. In 
degree (the block diagonal): 

j-l|j-2,l_f 3,ol 1 2 5 21J.3 

<^o = JiOL + J201 +jfi,2a +6u!i, UJ2 = a , ujq = a —^jia 

n 012 2,5 1,6 0,7 2 3 3 4 4 6 

U = U!q — LJi — UJ2 = ~T~ ^5 ~ + ~ ^0 ~T~ ^7 — ^1 ~ ^3 ~ ^4, ~ ^3 ~ ^4, ~ ^5 ■ 

Above the fj are functions on the (reduced) frame bundle, and the fj^a are the a"-coefficients 
of dfj. In degree 1: 



2 ' 

'^7 = -i/i.ia^-/32, /3i = = -fwl = -2a;| = = 



In degree 2: 

W4 = - 1 /i,i - I Pi ^1 = ^ /i,i - I Pi - i 83 ujI = j fi-2 + i f ,-, a'^ - 0, 

oj* = pia^ , i^l = 1^6 = -| /i,i - \pia^ - \ l3-j, cuj = - 6pi q'"* - ^ Pa , 

withpi = (/i /i,i-|-/i,i,i)/24, where /i^i^i is the a^-coefficient of d/i,i. Note that appears 
below in the Fubini cubic r^. In degree 3: 

= -4pi - p3 a'^ uj'i = -pi - i /i,3 - 2p2 - | /i Pi 

'^7 = -Pi + i /i,3 + P2 + I /i /3i a;| = -pi -|- ps , 

with p2 = (/i,i /i,2 + 4 /i,i,3)/48 and = (/i,i' - 2 /i^ /i,i - 2 /i /i,i,i)/48. And the final 
degree 4 1-form is 

u)-j = -4p3a^— 4p2a + P4 a — | /i,i /3i , 

where is a polynomial in /i, /2, /i,a, /i,i,i, /i,i,3 (homogeneous of degree 4 in derivatives). 
The Fubini cubics are 



r 



^ - -2a2^ + 2pia3^ = 3 ai^ 02 - | /i a2 as^ - ^ /i,i as^ 



1 

= — 3Q;ia2 r = 0. 
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These varieties are intrinsically flat (cf. §9.4). 

9.3. Case 3. Assume that = o? mod a^^o?; and iJ\ = mod Q;^,a^. The second 
tableau (after two sets of restrictions on our parameter space to remove torsion) has charac- 
ters (si, S2, S3) = (15, 2, 0). The second (reduced) prolongation has dimension 14. The third 
tableau (after one set of restrictions to remove torsion) has characters (si, S2, S3) = (11, 2, 0). 
The third prolongation is (also) of dimension 14. The fourth tableau (no torsion) has char- 
acters (si, 52,53) = (12,2,0), and the fourth prolongation is of dimension 16. Thus the 
system is involutive by Cartan's test. 

The Maurer-Cartan form (the components of non-negative degree) is given below. In 
degree 0: 

1 r l,/-2, 3 2 1, 2, 3 1 2 2 1 

ojI = a" + i(2/l-5l)Q^ = Q^ + i(/2-25i)a^ 

n 012 1|6 2,5 3 3 4 4 0,7 

U = UJq — U)i — UJ2 = Ull + IjJfi = U)2 ~r ^5 = ^3 = ^1 = ^'i = = '^0 -r ^7 ■ 

The /j, Qj, 1 < j < 2, are functions on the frame-bundle; the fj^a and gj^a are the a"'— 
coefficients of d/j and dgij ; and d/i = /?!, d/2 = — 2/32,d(7i = — 2/9iandd5f2 = /32 mod the 
semi-basic a"". Finally = -l + fi (/2-52)+/2,i-/i,2 andp2 = l + 52+52,1 -5i,2- 

In degree 1: 

ujI = -\ (2 /i - 5i) -132 = -I (2 /i - 51) - § /32 = -\ (/2 - 232) a^+P3a'-\ A 

'-'1= f (/2-2 52)a'-|/3i o-i = -i(/2-2fl2)a'+/3i u;| = i(2 /i - 51) a' - P4 a' - f /32 

= -i(/2-252)ai-h/3i ujt = -\{2h-gi)a^ -02 = p^a^ - 02 oj^ = -ps + 01 . 

Above, P3 = i (2 /i -51) + i 52 (/2 -2 32) + i (/2,2 - 2 52,2) and ^4 = i (/2 - 2 32) + i /i (2 /i - 
51) + i (2/1,1 - 51, i)- In degree 2: 

1^4 = P3 + i P4 + hi , cu'^ = ^ ps + ^ P4 + /i2 Q'^ - | ,53 , lut = - ^hi a* 
= i (2/1 - ffi + 2P3) a' - i (/2 - 252 + 2p4) + /ti a' - \ Pa 
We = i (/2 - 2ff2) a' - (2pi +P2) -l-ps - /3i , w| = (pi - 2^2) a' + i (2/i - 31) +pg ^3 _^ 
a;i = -i (2/1 -gi + 2p3)ai -h i (2/2 - 32 -h2p4)a^ -h fti - i /Js . 

In degree 3: 

Lol = 2p6q' + (|/ii + t|(2/i-.9i)(/2-232)) a^+P7a'-|(/2-252)/32 
c^g = (Ui-i|(2/i-(7i)(/2-2g2)) Q'-2p5a2+P8a^ + |(2/i-3i)/3i 
'^7 = (|/ii+ j^(2/i-5i)(/2-2ff2)) ai+p5a2+p9a'-i(2/i-5i)/?i 
w| = P6ai-h(|/ii-Fi(2/i-3i)(/2-252)) a^-fpioa^-hH/2-252)/32 

The P7,--- ,Pio above are polynomials in {fj,gj, hj, fj^k, 9j,k, hj,k}i<j,k<2- Finally, in degree 
4: 

= pii +pi2a^ + {^ h2,3 + {pi + P2) fe) - i (/2 - 2 32 - 8^4) 01 + \{2 fi - gi + Aps) 02 ■ 

The pii,pi2 above are polynomials in the {fj , gj , hj , fj^k, gj,k, hj^k}i<j,k<2- 

Finally, we remark that the functions fj , gj , hj satisfy a system of partial differential equa- 
tions. For example, (/2 + 2 (72)1 = (2 /i + 51)2- One should think of this system as reducing 
the freedom of our initial data from four functions of 14 variables to the two functions of 
two variables and twelve functions of one variable and specified by the characters. 
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The Fubini cubics are 

= -2{ai^ + a2^)-^hia3^ 
= 3ai2^2 + i(2/i -5i)a2a3^ + 2p4a3^ 
= -3ai + I (/2 - 252)01 "3^ - 2p3a3^ 
= 0. 

In r^, /ii is the term that first appeared in a;^ above. 
The intrinsic invariants (cf. §9.4) are 

Ji = -|(2/l-5l)-^3/2-52)52-|/2,2 + |^>3 
J2 = -|(/2-252)-|/i(/i + 25i)-^/i,i-|p4. 

9.4. Intrinsic invariants. Let Y'^ C be a 3-foId with frame-bundle J^neg on which the 
Maurer-Cartan form is as given in (9.1). From (9.2) we see that ¥{v G T^X \ F2{v,v) = 
0} C PT_i -E consists of exactly two points. These two points define a pair of line bundles 
which span the contact hyperplane T^i^x C TxX. 

Let's consider the situation more generally. Let M be a complex 3-dimensional manifold 
(not necessarily projective) admitting two line bundles Li and L2 such that H := Li (B L2 
is a contact distribution. Consider the bundle ^ M of all frames {61,62,63} of TxM 
such that ej spans Lj, and given any local section e : U G M ^ J^, [ei(x), 62(0;)] = 63 (x) 
mod Hx- Then ^ is a principle G-bundle with fibre group G C GL3C 

a, b G C , 
a, /3 € C\{0} 

Given 6 = {cj} G J-x and v G T^J-, the canonical semi-basic C^-valued 1-form on J- is 
given by 7r^{v) = if{v)ej. There exist connection 1-forms {0'^}s=i on such that 

dr]^ = r]^ A + 7]^ A 

drj^ = -Tj^ A 7]'^ + 7]^ A {9^ + 9^). 

Notice that this connection has torsion (see drj^). In fact, there are no torsion-free con- 
nections on J^. The family of connections preserving the structure equations above is 
4-dimensional. 

The choice of connection may be further refined (leaving one degree of freedom) so that 

d9'^ = 29"" Atj" + 9^ A7f - (p A 7]^ 

d6|2 = -9^ A7]^ -2 9^ At]'^ - <j)A jf 

d9^ = 9"^ A9^ -^At]^ + Ji7f A7f 

d^^ = 9^ AO* -4>Ari^ + J2r]^ Arf . 

Above, is a 1-form, and the J are functions on J^. 

The functions J are relative invariants. They define invariant tensors J\ (w^Aw^) (8) e^, J2 (w^ A 
0;=^) ®^ e r(M, f\^{T*M) (g) {TM/H)*). 
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Applying the above formulas to Case 2 of 9.2 shows the two invariants Ji and J2 are 
zero, so the integral manifolds are intrinsically flat. Similarly, we see Case 3 (§9.3) is not 
intrinsically fiat. 

9.5. Order three rigidity of . It is also true that Iq C Ipuhs but this is only seen to 
hold after one calculates a derivative. One can either prove third order rigidity of this 
way or cite [23]. 



10. Remaining rigidity proofs 

10.1. The varieties t^dCF"), Vd{Q'^) and Seg{^^ X P^^x • • • X P°"). Assume through- 
out this section that n > 1. 

Proof of Theorem 1.3 (a). Begin with the Veronese variety, and consider the Fubini system 
of order d + 1. We will show it is a priori more restrictive than the (/q, Jq) system; thus 
the (/q, Q) system being rigid implies it is rigid. Here — S^T* , the F^^k are the identity 
maps for k < d and all the F^^k = for s < k and Fg = for q > d. Since the grading 
is three step, the osculating filtration coincides with the Lie algebra filtration (albeit with 
different integers attached to the filtrands). 

The vanishing of the -Fjt,fe_i, k = 3,...,d fixes (8) in terms oiT* ®T and L* (g) L, thus 
together they fix qI{U)q to be Qq. The vanishing of the -Fjk,fc-2, k = A, ...,d fixes iS) N^-i 
in terms of L* T and the remaining forms in the system fix all other components of the 
Maurer-Cartan form taking values in the remaining spaces below the diagonal to be zero. 
Thus we are reduced to the {Iq, Jq) system, and Theorem 1.3 (a) follows from Theorem 1.9. 

□ 

Remark. Note that we only used a small part of the Fubini system to prove rigidity in this 
case. 

Remark. In in the case d = 2, n = 1, V2{'^^) C is a plane conic. Monge showed that 
the plane conies are rigid at order 5. This is consistent with the Lie algebra cohomology: 
In this case we have g = U = S'^C'^ = U2wi- '^^^ decomposition sl(C/) = ® ^4u)i yields 
0-*- = U^i^^. The cohomology group H^{q^,q^) is one-dimensional, with weight —&lo\ as a 
00 = C module. Since the grading element Z for U is given by Z{(jji) = ^, we see that 
i?3(0_,0-'-) / 0, an obstruction to Fubini rigidity at order four. 

Proof of Theorem 1.3 (h). The Vd{Q) Fubini system is similar to the Veronese system. All 
Fubini forms are zero except for the fundamental forms and they are given by N]^ ~ gkj^* 
for k<d, Nd+i = go5<^-ir*, Nd+2 = Q'^oS'^-'^T* N2d = Q'^ and all higher fundamental 
forms are zero. Here F2d+i,2d fixes the ® Nd component of the Maurer-Cartan form in 
terms of the T* ®T ® L* ^ L-component, and similarly down the line. Thus Theorem 1.3 
(b) follows from Theorem 1.7. □ 

Proof of Theorem 1.3 (c). We merely sketch the proof for the Segre variety Seg{¥^ x 
P"2 X • • • X P"' ) - it is similar to the two above. The r + 1 Fubini system implies the (Iq, Jq) 
system. We compute that H^{q^,q^) = for d > 1. Thus the (Jq, Jq) system is rigid. □ 
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10.2. Es/Pi C p3874_ establish that E^/Pi is rigid to order five, it suffices to show 
that the osculating sequence has length four and that the a;g±^ component of the Maurer- 
Cartan form is forced to vanish on integral manifolds of the fifth order Fubini system. The 
first assertion follows because the shortest sequence of negative roots taking uji to —uji 
is of length four. The second by arguments similar to the adjoint case for the T!:^ (g) r_i 
component, forcing this component of the Maurer-Cartan form to be the spin representation 
of D-j and by observing that the fourth fundamental form is the quadric on T_2, forcing the 
rj^2 ®T-2 component of the Maurer-Cartan form to be the standard representation of Dt. 
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